Network traffic anomalies such as failures and attacks are common in today's network. Identifying network traffic anomalies rapidly and accurately is important for large network operators. Estimation of the connection degree identifies hosts that are responsible for anomalies like DDOS (Distributed Denial-Of-Service), flash crowds and network failures.
Depending on traffic volume and link speeds, a suitable detection method has to be selected to identify network anomalies. Recent research efforts have been directed towards developing scalable heavy hitter detection techniques for accounting and anomaly detection purposes. Heavy hitter techniques do not correspond to flows, experiencing significant changes in the network traffic.
Researchers have developed various systems like Snort and Flowscan, for example. Snort is network intrusion prevention and detection system which utilizes rule based language combining signature, protocol and anomaly inspection methods.
Flowscan analyzes and reports on flow data exported by Internet Protocol routers. Flowscan groups PERL scripts and elements such as flow collection engine, a high performance database, and a visualization tool. After all the tools are assembled, the Flowscan system produces graphic images, appropriate for use in web pages. Flowscan provides a continuous, near real-time view of the network traffic through a network's border. However, these systems are not suitable because of the massive amount of network traffic in high-speed links.
Further, data monitoring algorithms based on efficient data structures have been in use for high traffic user detection and traffic-volume queries. The data monitoring algorithms allow monitoring of data network traffic without tracking data individually for each separate key. The data monitoring algorithm utilizes parallel hash tables to identify large flows using a memory that is only a small constant and larger than the number of large flows. However, this technique only detects high traffic users and does not detect users having significant changes in traffic.
Another approach in identifying network anomalies includes using data streaming techniques. The data streaming techniques locate hosts with large connection degrees. These techniques use a reversible connection degree sketch to monitor network traffic anomalies. They may use bit array operations and hash functions based on Chinese-Remainder Theorem (CRT) to create a connection degree sketch of network data stream. However, the quality of the results may be reduced by false positives and false negatives.